Simplified User Interaction with Intrusion Systems Based on Identified Presence Detection

ABSTRACT

A unified presence detection and prediction platform that is privacy aware is described. The platform is receives signals from plural sensor devices that are disposed within a premises. The platform produces profiles of entities based on detected characteristics developed from relatively inexpensive and privacy-aware sensors, i.e., non-video and non-audio sensor devices. The platform using these profiles and sensor signals from relatively inexpensive and privacy-aware sensors determines specific identification and produces historical patterns. Also described are techniques that allow users (persons), when authorized, to control remote devices/systems generally without direct interaction with such systems merely by the systems detecting and in instances predicting the specific presence of an identified individual in a location within the premises.

CLAIM OF PRIORITY

This application is a Continuation application of and claims priorityunder 35 U.S.C. §120 to U.S. patent application Ser. No. 15/151,613filed May 11, 2016, entitled: “IDENTIFIED PRESENCE DETECTION IN ANDAROUND PREMISES”, the entire contents of which are hereby incorporatedby reference, and which in turn claims priority under 35 U.S.C. §119(e)to provisional U.S. Patent Application 62/160,772, filed on May 13,2015, entitled: “IDENTIFIED PRESENCE DETECTION IN AND AROUND PREMISES”,the entire contents of which are hereby incorporated by reference.

BACKGROUND

This description relates systems that detect presence of entities in anenvironment through electronic sensors.

It is common for persons to interact with electronic devices thatrequire user input to control. Such user devices are ubiquitous anddetermining presence from such interactions are relativelystraightforward. One example is a portable electronic device with GPS.With such devices it is possible to detect the location and hence in asense presence of an individual in a specific location.Commercial/residential surveillance and/or intrusion and/or alarmsystems have sensors to detect the presence of conditions at a premisesthrough processing of various types of signals from detector/sensordevices either by a local processing station, e.g., an intrusion paneland/or at a central monitoring station. Moreover, commercial/residentialsurveillance systems in particular include electronic cameras, e.g.,video cameras that send data such as video data from such cameras to,e.g., a central monitoring station, where such data can be analyzed.

Such sensors are generally low cost and often prone to producingconditions that can result in false alarms.

SUMMARY

One of the major limitations of traditional approaches to detectingpresence is that absent relatively sophisticated techniques and/orspecific user interaction with systems, it is generally notcost-effective in many environments, e.g., a home, to detect presence ofspecific individuals and recognize the specific individuals presence byuse of relatively inexpensive sensors, such as found in alreadyinstalled surveillance and/or intrusion and/or alarm systems or as abyproduct of such surveillance and/or intrusion and/or alarm systems. Inaddition certain types of approaches, e.g., video and audio approachescan be rather intrusive from a privacy perspective.

Aspects include methods, systems and computer program products stored onphysical, computer readable hardware devices that can include varioustypes of computer storage devices including memory and disk storage,features of which are set forth in the claims.

An aspect involves a system to determine specific presence of anidentified entity, the system including a plurality of non-video andnon-audio sensor devices disposed in a premises, a computing systemcomprising a processor and memory that are configured to receive signalsfrom at least some of the plurality of sensor devices in response anentity moving through the premises and retrieve profile records ofentities having established profiles on the system, with each profilestoring corresponding identification information of an entity associatedwith the profile and parameter information of characteristics associatedwith the corresponding entity. The system for each profile is configuredto determine from the signals received from the sensors correspondingparameters, determine a number of sufficiently close matches of thedetermined parameters to parameter information from retrieved profilerecords, and select from the retrieved profile one of the profileshaving the highest number of determined number of sufficiently closematches as the specifically identified entity.

The following are some additional features of the above aspect.

The system is further configured to process the signals from the sensorsto determine presence of a moving object and to determine parametervalues associated with the moving object. The system is furtherconfigured to produce a profile record populated with the determinedparameter values associated with the moving object, when each of theretrieved profile records have no matching parameters. The sensor devicetypes are selected from the group consisting of motion detectors, glassbreak detectors, noxious gas sensors, smoke/fire detectors,contact/proximity switches, temperature sensors, vibration sensors, airmovement/pressure sensors, chemical/electro-chemical sensors, weightsensors, global positioning system transceivers, optical detectors,biometric sensors, EGG/Heartbeat sensors in wearable computing garments,network hotspots, and r.f. detectors. The computing system is furtherconfigured to determine which profile record or records have above athreshold number of closely matching parameter values, and when pluralprofile records have above the threshold number of closely matchingparameter values, determine which profile record has a highest number ofmatching parameter values. The profile record is a first profile recordand the computing device is further configured to receive a request togenerate a profile, receive user inputs of user personal information,produce a second profile record from the data received from the user,and store the second profile record.

An additional aspect involves a system including plural sensor devices,a computing device, configured to receive signals from the plural sensordevices, retrieve a profile of an entity from a data store, retrievemovement flow records from the data store, the movement flow recordscomprising data from one or more sensors and temporal informationregarding time and day of collection, analyze the retrieved movementflow records according to a historical pattern rule to determine fromthe sensor signal data and the temporal information, a set of sensorsthat satisfy for the retrieved profile the historical pattern rule,construct a representation of the set of sensors that corresponds to anorder and relationship of sensors encountered by the entity associatedwith the movement records.

The following are some additional features of the above aspect.

The representation is a graph structure that includes a set of nodescorresponding to sensors and a set of connecting lines that establishrelationships between the nodes. The representation is a graph structurethat includes information associated with each node and edge of thegraph. The computer system is further configured to produce anhistorical pattern record. The computer system is further configured toproduce an historical pattern record that includes a reference to therepresentation of the set of sensors, a profile ID corresponding to theprofile and a Rule ID. The produced historical pattern record furtherincludes a data field comprising supplementary data. The analysisapplies a pattern recognition algorithm to analyze the retrievedmovement flow records to determine a historical pattern from themovement flow records. The system is further configured to determine alocation of a user from signals sent from one or more of the sensordevices, determine from signals sent by the one or more sensor devices,an identity of the user, retrieve from the database one or more patternscorresponding to an action predicted by the system to be performed bythe identified user, and send control signals corresponding todetermined control actions to the one or more systems/devices to performthe determined control action by the systems/devices. The system has afirst one of the one or more control devices is a panel for analarm/security system, the system further configured to authenticate theuser for accessing and performing the control action on the panel. Thesystem is further configured to process received sensor data from thesensor devices to establish identity of a specific person in thepremises, retrieve information regarding restrictions placed on thespecific individual in the premises, process sensor data from one ormore of the sensor devices against restrictions to determinenoncompliance with the restriction, and form a message regardingnoncompliance with the restriction to send to a user device. The datauses presence information collected from sensors for pet detection ormonitoring family members. The system is configured to receive a requestfor operating in an activity replication learning mode for one or morespecific periods of time. The constructed representation represents ahistorical pattern of actions performed by one or more individuals overspecified one or more periods of time, with the system configured toproduce a listing of actions performed by the one or more individualsover the specific one or more periods of time, produce control messagesaccording to the listing to control devices and or systems within thepremises, and send the control messages according to a sequence in thelisting to control the devices and or systems within the premises.

An additional aspect involves a system for determining occurrences offalse alarms triggered by an intrusion detection system. The systemincludes a computing device including a processor device and memory, thecomputing device configure to receive data from a plurality of non-videoand non-audio sensor devices, receive an assertion of an alarm conditionfrom an intrusion detection system, process the received sensor datafrom the plurality of sensor devices to detect presence of a movingobject within a specific location, retrieve from a database profilerecords of expected entities in the premises, determine one or morematches of physical attributes from the profiles to detected physicalattributes provided by detection of presence from the sensor data,determine from presence or absence of matches whether to confirmassertion the alarm condition from the intrusion detection system, andsend the determined confirmation to the intrusion detection system.

The following are some additional features of the above aspect.

The system is configured to produce by a profile system a new profilethat is populated with data received from the plurality of non-video andnon-audio sensor devices, when there are no matches. The system isconfigured to send a message to the intrusion detection panel systemwhen confirmation of an alarm condition is determined to cause theintrusion detection system to issue the alarm. The system is configuredto provide an indication of what triggered the alarm. The system, basedon the processing of the profile and presence information, determinesthat the alarm was triggered by movement of a family pet or the presenceof authorized persons in the premises the system, sends to the intrusiondetection system a message to retire the assertion of the alarm. Thesystem, based on the processing of the profile and presence information,confirms the assertion of the alarm when the system detects the presenceof unidentified or unauthorized individuals. The determined confirmationis used an input in processing of potential alarm conditions by theintrusion detection system. The system stores the event for subsequentassertion based on a confirmation from other sensors.

An additional aspect involves a system including an intrusion detectionsystem that includes a computing device that receives data from pluralnon-video and non-audio sensor devices, the computing device configureto detect intrusions into a premises being protected by the intrusiondetection system, a presence detection system operatively coupled to theintrusion detection system, the presence detection system configured toidentify a current location of a user in the premises, retrieve storedprofile data and historical pattern records associated with the profile,retrieve premises profile data that specifies locations of varioussensors, layouts of rooms, equipment in the premises, devices/systemsthat can be electronically controlled, IP addresses of suchsystem/devices, determine actions that the user is authorized to performto control the intrusion detection system and other equipment, devices,systems in the premises based on the determined location, the retrievedhistorical records and the retrieved profile records.

The following are some additional features of the above aspect.

The system is configured to send actions that control the otherequipment, devices, and systems to a user device. The system isconfigured to determine mechanisms available in the current location bywhich the user can communicate with the presence detection system andenable determined mechanism in the determined location to communicatewith the presence detection system. The system is configured todetermine identification of the user by stored profile records andreceived sensor data for configuring the intrusion system. The system isconfigured to determine user activity with the determined user activityproviding a trigger event to enable user interaction with the intrusiondetection system. The enabled user interaction with the intrusiondetection system is by voice commands from the user. The system includesa plurality of sensor devices that send the sensor data to the system.

One or more of the above aspects may include one or more of thefollowing advantages.

By combining low cost non-video and non-audio sensor technologies, someaspects can detect presence of specific identifiable individuals, andyet maintain a notion of privacy. In some embodiments, data may leavethe person's premises, however, if there is a full streaming audio ofconversations (or video of movements) the person might not be so keenabout sharing such data. Thus some aspects use a combination oflower-cost technologies (or in some instances higher cost but lessintrusive technologies) and signals from these types of sensors on theirown may not raise significant privacy concerns.

Aspects include methods, systems and computer program products stored onphysical, computer readable hardware devices that can include varioustypes of computer storage devices including memory and disk storage,features of which are set forth in the claims.

The details of one or more embodiments of the inventions are set forthin the accompanying drawings and the description below. Other features,objects, and advantages of the invention are apparent from thedescription and drawings, and from the claims. Various use cases arediscussed herein.

DESCRIPTION OF DRAWINGS

FIG. 1 is a schematic diagram of an example of a security system at apremises.

FIG. 2 is a block diagram of a presence detection/prediction andprofiling system for detecting presence and building profileinformation.

FIGS. 3A-B are flow charts of a profile building process.

FIG. 4 is a flow chart of updating a profile using data generated by adetection system.

FIGS. 4A-D are block diagrams of various exemplary records.

FIGS. 5A, 5C and 5D are flow charts of processes executed on a presencedetection and prediction system.

FIG. 5B is a graph structure.

FIG. 6 is a flow chart depicting historical pattern discovery.

FIG. 7 is a flow chart depicting an example of historical patterndiscovery.

FIG. 8 is a flow chart of a process for detecting and recognizingpersons using inexpensive, privacy aware sensor technologies.

FIG. 8A is a flow chart detailing one of the processes of FIG. 8.

FIG. 9 is a block diagram of an integrated, unified intrusion detectionand presence detection/prediction/profiling system.

FIG. 9A is a flow chart detailing one of the processes of in the systemof FIG. 9.

FIG. 10 is a flow chart detailing user interaction.

FIGS. 11A and 11B are flow charts detailing activity replication.

FIG. 12 is a flow chart depicting monitoring.

FIG. 13 is a flow chart for detecting alarm events in the integratedintrusion detection and presence detection/prediction and profilingsystem platform.

DETAILED DESCRIPTION

Described below are techniques that allow users (persons) to controlremote devices/systems generally without direct interaction with suchsystems merely by the systems detecting and in many instances predictingthe specific presence of an identified individual in a location. As usedherein “specific presence of an identified individual” is defined as theunique presence of an identifiable specific individual that is generallyrecognized by systems described herein as such described systems aretrained. In some implementations/embodiments “specific presence of anidentified individual” is further defined by a set of sensorcharacteristics, which set excludes video and audio sensors, and asystem stored profile that is either manually populated by a userentering data into a system or is automatically populated overtime byprocessing of various sensors excluding video and audio sensors.

For purposes of explanation, sensors in a user's house produce signalsthat are temporally analyzed by the disclosed electronic systems todevelop patterns of behavior. These electronic systems in turn are usedto control many features in the house, merely by having detectedpresence and in certain embodiments, detection of specific presence,i.e., the “specific presence of an identified individual.” That is theelectronic systems identify presence of a specific individual, beingable to distinguish presence of a mother for instance from a child or afather or a visiting adult female.

Some of the many features that can be controlled include locking andunlock doors, arming and disarming a home security system, controllingremote media systems, such as radios, televisions, controlling openingand closing of drapes/blinds, evaluating the condition of mechanicalequipment within a premises, detecting ambient environmental conditionsthrough corresponding sensors and based on such detection modify theenvironment upon detection of specific presence. For instance, anambient environment sensor, e.g., a light meter could detect low lightconditions and the PDPS 40 through a connected lamp having a light-bulbturn that lamp on, and so forth. Constant analysis of an audio signatureof a home/facility over time may help detect the onset of changes and/orproblems in a facility such as a home or business.

Unified Presence Detection and Prediction Platform

Referring now to FIG. 1, an arrangement 10 including a security system12 at a premises 14 is shown. In this arrangement 10, the premises 14 isa residential house, but the premises may alternatively be any type ofpremises, e.g., commercial buildings, industrial buildings, campuses,etc. The security system 12 includes a control panel 16,sensors/detectors 28 and a keypad 30. The security system 12 is incommunication with a central monitoring station 18 and one or moreauthorized user devices 20 (only one shown) through one or more datanetworks 24 (only one shown), such as the Internet. The security systemcan be any one of fire, alarm, access control, surveillance, physicalintrusion, etc. types of systems or can be an integrated system thatcomprises any combination of two or more of such systems.

While the arrangement 10 is described as part of a security system, insome implementations, the arrangement need not be a security system perse but need only be a computing system that receives sensor data andincludes processing algorithms to detect specific presence, generatesmessages based on detected presence, and that outputs the messages toother electronic systems control or otherwise interact with othersystems. These other systems can be local and/or remote systems. As anexemplary implementation, however the arrangement will be described inthe context of an integrated security system 12 (referred to hereinafteras security system).

In the security system 12, the control panel 16 is in communication withone or more detectors/sensors 28 (many more of which would likely beused than those shown in FIG. 1) and receives information about thestatus of the monitored premises from the detectors/sensors 28. Examplesof detectors/sensors 28 (hereinafter sensors detector are usedinterchangeably) include motion detectors, glass break detectors,noxious gas sensors, smoke/fire detectors, contact/proximity switches,video sensors, such as camera, audio sensors such as microphones,directional microphones, temperature sensors such as infrared sensors,vibration sensors, air movement/pressure sensors,chemical/electro-chemical sensors, e.g., VOC (volatile organic compound)detectors, weight sensors, LIDAR (technology that measures distance byilluminating a target with a laser and analyzing the reflected light),GPS (global positioning system) receivers, optical, biometric sensors,e.g., retina scan sensors, EGG/Heartbeat sensors in wearable computinggarments, network hotspots and other network devices, and others.

Some of these sensors 28 such as motion detectors, video cameras, glassbreak detectors, noxious gas sensors, smoke/fire detectors, microphones,contact/proximity switches, network hotspots and other network devices,can be found presently in homes, and are either relatively inexpensiveand/or relatively common. Others such as vibration sensors, airmovement/pressure sensors, chemical/electro-chemical sensors, VOC,weight sensors, GPS receivers, optical, biometric sensors, e.g., retinascan sensors, EGG/Heartbeat sensors in wearable computing garments,LIDAR, and others would be rarely found in most residential environmentstoday, but may be in commercial/industrial environments.

The sensors 28 may be hardwired to the control panel 16 or maycommunicate with the control panel 16 wirelessly. The sensors 28 sensethe presence of a change in a physical condition, whether the changeinvolves motion, glass breakage, gas leaks, fire, breach of an entrypoint, temperature, sound, weight, pressure, chemical, among others, andsends sensor information to the control panel 16. Based on theinformation received from the sensors 28, the control panel 16determines whether to one or more trigger alarms, e.g., by triggeringone or more sirens (not shown) at the premises 14 and/or sending one ormore alarm messages to the monitoring station 18 and/or to a user device20, as would be conventionally done with existing security systems.

However, the control panel 16 also includes hardware and software(collectively referred to as a “presence detection and prediction system40” that detects and predicts presence of specific individuals andgenerates messages that control local and remote systems and/or whichproduce “specific person presence data” that can be used in manydifferent contexts as will be discussed below.

A user may access the control panel 16 to control the security system,e.g., disarm the security system, arm the security system, enterpredetermined standards for the control panel 16 to trigger the alarms,stop the alarms that have been triggered, add new sensors, change sensorsettings, view the monitoring status in real time, etc. The access canbe made directly at the premises 14, e.g., through a keypad 30 connectedto the control panel or with other techniques. In some implementations,the control panel 16 may also include a display (not shown) that shows agraphical user interface to assist a user's control of the securitysystem. The display may be a touch screen such that the user mayinteract with the control panel and the security system directly throughthe display.

The user may also access the control panel 16 through the user device20, which can be at or be remote from the premises 14. To allow a userto access the control panel 16 through the user device 20, and toprotect the security system from unauthorized accesses, the controlpanel 16, the monitoring center 18, and/or the user device implementsone or more levels of authentication, including user biometricauthentication. The authentication(s) can also be based on input from auser, such as a security code or a PIN provided to the user, a passwordcreated by the user, and/or an RFID chip provided to the user or aspecific gesture or combinations of any of the above.

Presence Detection and Prediction System

Referring now to FIG. 2, a presence detection and prediction system 40is shown. The presence detection and prediction system 40 is shown aspart of the security system 12 inside the control panel 16, but asmentioned above could be a standalone system. The system could be alocal system, e.g., on the premises or a remote system. In addition,some aspects could be configure to be locally deployed whereas otherscould be remotely deployed and one or more aspects of the presencedetection and prediction system 40 could be duplicated, meaning that oneinstance could be locally deployed whereas another could be remotelydeployed. The presence detection and prediction system 40 includes aprofiling system 42 for processing sensor signals to build profiles anda detection and prediction system 60, which as shown can share circuitrywith the profiling system 42.

Profile System

The profiling system 42 includes a user interface 44 that allows a userto input data directly into a profiler 46. In addition, to the userinterface 44 and the profiler 46, the profiling system 42 also includesa network interface 48, a sensor interface 50 that receives signals fromone or more various sensors 28, and profile storage 52. The profilingsystem 42 connects to a network 50 over which the profiling system 42sends/receives information from one or more remote databases 49 (orother systems such as servers not shown). A user can also access theprofiling system 42 via a remote or local device, e.g., keypad orkeyboard or voice actuation to input person information that will becomepart of a presence profile, as will be described below.

Detection and Prediction System

The detection and prediction system 60 includes a detector 62 thatreceives sensor signals, in some implementations accesses profiles fromlocal storage in the profiling system 42, and executes one or moredetection modules to detect specific presence of one or more specificentities. The detection and prediction system 60 also includes a userinterface 64 that allows a user to input data directly into a detectionsystem 62. In addition, to the user interface 64 and the detector 62,the detection and prediction system 60 also includes at least one andgenerally several device interfaces 68. The device interfaces 68 receivecontrol signals generated by a control signal generator 70 andconfigures such control signals to be in an electrical and logicalformat according to the type of device and/or system to which a specificcontrol signal is sent.

The control signal generator 70 receives inputs from the detectorcircuitry 62 and a predictor module 72 and access rules. Control signalsmay or may not be generated depending on outcomes of execution of therules by the control signal generator 70. Generally, data from thedetector circuitry 62 and the predictor module 72 are inputs the rules.The predictor module 72 also receives inputs from the sensor interface50, so as to receive sensor signals from one or more of the varioussensors 28, and receives data from profile storage 52. The predictormodule 72 also accesses stored detected historical patterns that are ormay be associated with stored profiles.

Building Profiles

Referring now to FIGS. 3A and 3B, modes of operation of the profilesystem 40 are shown. In FIG. 3A, a user produces 70 a profile for anentity, which profile includes personal information sufficient to enablethe presence detection and prediction system 40 to detect the specificpresence and a type of identity of an individual entity during operationof the presence detection and prediction system 40, as will be discussedbelow. Identity can be a legal identity, e.g., a user's given name oridentity can be a system defined identity, e.g., a system generatedtracking number or label.

A user inputs into the profiling system 42 a request 72 to produce aprofile. The profiling system 42 receives the request to generate aprofile and presents 74 an interface, e.g., renders a graphical userinterface (GUI), not shown on a display or otherwise has an interface,e.g., a keypad and simple display that are menu-driven to allow the userto input information for the profile. The profiling system 42 opens 76 anew profile record and receives 78 data from the user. The profilingsystem 42 stores 80 the user populated profile record in profile storage(52, FIG. 2) and/or sends 82 via the network the profile record to theremote database (49, FIG. 2) accordance with privacy indicators.

Profiles can be established for residents of the structure, frequentvisitors, pets, and even certain people identified as individuals, whomshould not be on the premises and whose presence may for example triggeran alarm, even when the security system 12 is otherwise not armed. Eachprofile can be so indicated.

The information that becomes part of the profile record includes some orall personal information such as name, age, weight, gender, height, eyecolor, hair color, etc., as well as type of entity, e.g., resident,frequent visitor, pet, etc. In some implementations, the personalinformation that is input is accompanied by a privacy indicator thatdetermines whether that piece of information is shared with othersystems, e.g., whether it leaves the local profile store. In general,the profiling system 42 may store many different profiles on manydifferent persons, especially family members and frequent visitors in aresidential setting or employees in a work setting.

In FIG. 3B, generation of a profile record is system-initiated. Theprofile system 42 receives 91 a request to produce a profile for anentity, which profile will include personal information sensed by thepresence detection and prediction system 40 for a specific unidentifiedindividual entity collected during operation of the presence detectionand prediction system 40, as discussed below. In this instance, identityis a system defined identity, e.g., a system generated tracking numberor label. The profiling system 42 receives 91 the request and receivesdata 92 from the presence detection and prediction system 40. Theprofiling system 42 opens and populates 94 a new profile record andprovides in the new record a system generated identity 96, e.g.,tracking number or label. The profiling system 42 stores 97 the systemgenerated profile record in profile storage (52, FIG. 2) and/or sends 98via the network the system generated profile record to the remotedatabase (49, FIG. 2) in accordance with privacy indicators.

Sensor Signals

Conventionally, video cameras and microphone technologies are wellsuited for detecting presence. Video cameras and microphones have beenused for presence detection. Video cameras and microphones produce videodata and audio data that are processed using recognition processing andanalytics. While excellent for presence detection and recognition, videoand audio technologies generally can present major privacy concerns formany users. Moreover, video cameras are relatively expensive comparedwith sensor types discussed below.

Electronic devices generally emit r.f. during operation, and some ofthese devices generally produce different r.f. signatures. When a personturns on a television the action of turning on the television canproduce r.f. energy that is emitted and which can be detected by an r.f.sensor. Such an r.f. emission can be used to detect presence. Similarly,other r.f. devices such as computers, cellphones, and the likeproduce/emit r.f. energy, especially when those devices are wirelesslyconnected to a network. These devices generally have a unique r.f.signature at least from connection information that involves an IPaddress in such r.f. devices attempting to connect to networks. Such asignature can be used to detect presence generally and specific presencein many instances.

Sound detection—Voice recognition can be used for detection. Howevervoice recognition and especially speech recognition may pose privacyconcerns. However voice signatures may pose fewer concerns and meresound detection even fewer concerns, and can be used for presencedetection. Sound signatures can be developed based on sounds people makeduring movements, e.g., a signature one makes walking up/down stairs orjust by the sound of a person's footsteps. There could be other audiosignatures from noise recognition to identify an activity that mightallow the system to infer an action.

Temperature—Presence can be determined through detection of whether thetemperature of room has changed (e.g., windows opening, wind of a bodypassing through it creating a draft, air movement, pressure), sunshining in a window because a shade is raised or a curtain is opened.Similar changes can be detected in humidity caused by a human enteringthe area.

Vibration—Vibration changes detected within a room can be used toindicate motion in the room and depending on the level of systemanalysis can also be used to identify violent struggles, such as couldoccur by kicking of a door vs. knocking on a door. Direction andmovement information can be obtained with a sufficient number ofvibration sensors.

Chemical/electrochemical—Chemical/electrochemical emissions/changes canbe detected when a person is in a room, and in some instances uniqueemissions can be used to generate a signature to identify who the person(or animal, e.g., pet) is by processing sensor data and determining abiochemical signature that can be relatively unique for a person oranimal. Such a unique biochemical signature can be used to identifysomeone new in a room or someone who came in the room in the past byretrieving a record of the biochemical signature recorded previously.VOC sensors can determine if there's smoke in the air, and organiccompounds. There exists various types of VOC sensors, such as formarijuana detection as well as cigarette, cigar, detection, and certaintype of shampoo, soap, etc. and are similar to the chemical signatures.

Contact—Contacts at entry and exit doors (and other places) can be used,to determine human presence as is commonly used in existing intrusiondetection systems to detect presence. For example, if a door or windowis opened, the system can infer that a human is present in the space.

GPS (global positioning system) transceivers. User devices such as, butnot limited to, cell phones, generally have GPS transceivers thatsend/receive location data of the device (as well as device identifyinginformation). The GPS messages can be used to detect movement, rate ofmovement and direction, especially outside of buildings.

Weight Sensors—Weight sensors, such as pressure or contact pads, can beinstalled in and under, furniture, carpeting, and flooring in thepremises. Weight can be used to estimate the number of people in anarea, whether people have fallen, (e.g., by using a distribution of padsand determine that the weight is distributed over a larger area). Weightcan also be used with processing weight signals in combination withother sensors to provide an indication of specific individuals within anarea.

Heat detection—Heat changes can be detected and used to estimate thesize, direction, a number of people. Heat changes can be used toestimate, whether a person has just arrived from outside, e.g., person'ssignature is extremely hot or cold from being outside.

Optical or retina scanning detectors can also be employed with thepresence detection and prediction system 40, to identify presence byspecific individuals.

ECG Heartbeat Detection—ECG Heartbeats detectors can be used to detectheartbeats, e.g., in wearable computing and by using Doppler Effectprocessing such detected heartbeat signals can be processed to estimatemotion towards or away from a another sensor by capturing smallvariations in the frequency of the heartbeats moving towards or awayfrom a sensor that is picking up heartbeats form sensors on the wearablecomputing, e.g., from fitness bands that monitor ECG signals andtransmits the ECG, signals to an, e.g., an r.f. sensor. The DopplerEffect is used to determine motion towards or away from a particularsensor by detecting small variations in the R.F. frequency of a carriersignal sent to the r.f. sensor from the ECG detectors. This will allowfor determination of motion direction toward and away from any detector,and motion towards the left and right or up and down if severaldetectors are used.

Light detection—sensors can detect shadows moving across a room ordetect whether a light has been turn-on (or off), determine if lightswere left on while people are no longer in the room.

Devices coming online—person portable devices connecting voluntarily(authorized, known devices) or involuntarily (unauthorized, unknowndevices). The presence detection and prediction system 40 may havedetected a unique device (through device IP address) having approachedthe residence from the outside several times over an interval of timeand now has detected that device inside the residence. Where the deviceis associated with a known person, this technique can be used foridentification in addition to presence detection.

The above physical/chemical/optical characteristics can be capturedusing generally conventional sensors. In many installed intrusiondetection (e.g., integrated security, intrusion, alarm systems) many ofthe above sensors are already deployed.

Using Presence Information Collected from Sensors for Movement FlowRecords

Referring now to FIG. 4, processing 100 for production of an updating ofmovement flow records executed in the presence detection and predictionsystem 40 (hereinafter also referred to as PDPS 40) is shown. The PDPS40 is deployed either in the control panel 16 (or as a stand-alonesystem) and receives 102 sensor data from various sensors 28. Thesensors 28 will generally be of several different types that detect anyof the aforementioned physical/chemical/optical characteristics.

The PDPS 40 processes 104 the sensor data to detect the presence of anindividual or entity in a location. One or more sensors send messagescontaining raw sensor data to the PDPS 40. In some embodiments, the PDPS40 is in communication with the sensors via one or more (wired orwireless) data networks that are typically local networks. In otherembodiments such as when the PDPS 40 is deployed in the panel 16, thePDPS 40 can receive sensor data from the panel 16. The PDPS 40identifies 106 a presence of a moving object in a room, by detecting oneor more physical changes in sensor data from one or more of the abovesensors.

When a presence is not detected, the PDPS 40 continues processing 104received sensor data, as such data is periodically received. Whenprocessing indicates a detection of a presence of an entity, the PDPS 40compares 108 characteristics of the sensor data that was process tocorresponding sensor data stored in retrieved movement flow records thatare associated with stored profiles to detect a specific presence, i.e.,to detect the presence of a specific entity. When a specific presence isdetected, the PDPS 40 has found one or more movement flow records havingsensor data that match 110 (or closely matches) sensor datacharacteristics of the presence that was detected. Details on matchingare discussed below.

If a matching movement flow record was found, the PDPS 40 retrieves 112the matching record (or records, if more than one record was found tomatch). The matching record may include a pointer to a stored profilerecord produced by the process discussed in FIG. 3A and the movementflow record will include or have references to historicalpattern-detected records (discussed below). Optionally, the matchingmovement flow record may be updated 114 and stored 116. Updating of anexisting record would generally be performed when a significant changein a value of the data (more than an empirically set thresholddifference) or new sensor data were produced.

If a retrieved movement flow record does not match, the PDPS 40 checks120 if there are more movement flow records to be examined, if so,retrieves 122 the next movement flow record and continues at compare108. If there are no more movement flow records and no matching movementflow record (or records) were found, the process generates 124 a newmovement flow record and generates a profile record using the process inFIG. 3B and stores 126 the new movement flow record. The new movementflow record may be updated at some point to include a pointer to aprofile record whether produced by the process discussed in FIG. 3A oras discussed in FIG. 3B. The new movement flow record will include orhave references to historical pattern-detected records (discussedbelow).

The process 100 uses the received sensor data to produce “movement flowrecords” that capture information regarding the detected specificentity's interaction with the premises as reflected by the sensor data.The movement flow record may be added to the profile if the movementflow record represents significantly new information pertaining to thatentity, as discussed below.

If a specific presence is detected, but a matching record was notdetected, the PDPS 40 generates the new profile record and the new“movement flow record.” At some point personal profile information asdiscussed in FIG. 3A can be inputted into the profile record andovertime the movement flow records are updated and at some point may beassociate with the updated profile. In many situations, the personalprofile information may not ever be made available to the PDPS 40, inwhich case the PDPS 40 generates the profile record merely with a uniqueidentifier, and stores the produced “movement flow record” with apointer to associate it with the PDPS 40 generated profile record.

The PDPS 40 associates attributes of the detected entity with thegenerated profile record, such that the profile record can added to thePDPS 40 associated with the movement flow record and if additionalmovement flow records by the detected entity are subsequently producedby that detected entity further interacting with sensors in the premisesthese additional movement flow records can likewise be associated withthe PDPS 40 generated profile record. The “movement flow record” isadded to this profile if the “movement flow record” representssignificantly new information pertaining to that detected entity, asdiscussed below.

The PDPS 40 is a ‘learning system’ where profiles are constantlyupdated, e.g., by adding references to movement flow records (and/orprofile records) and are used to determine identification of entities,e.g., people, by combining data from numerous low cost/low power sensorsthat are not generally capable individually to determine identity, whileat the same time avoiding use of sensors that can be privacy intrusive,such as cameras. The profiling processing can be done in someimplementations on location, e.g., in the PDPS 40 installed as astandalone unit or as part of an intrusion detection system, as well asin the “cloud.”

Sensors that are used include but not limited to motion, e.g., by motiondetectors, radio frequency emissions, e.g., short distance, low energycommunications, such as Bluetooth, Wi-Fi emissions (determine presencesby observing bandwidth traffic on a local network) as well as video,LIDAR, audio, temperature changes, air movement changes and pressurechanges.

With some embodiments chemical sensors can detect chemical emissions andproduce chemical signatures. LIDAR, can be used to sense movement. ThePDPS 40 can process one of more of these sensor signals to recognizeshapes and figures, e.g., by LIDAR and/or video.

Depending on what parameters from what sensors the PDPS 40 is measuringwill determine what processing is performed by the PDPS 40. Sensors ofvarious types can be placed in many different types of devices andappliances commonly found in the home. One of the richest sources ofdetecting presences is from wireless routers or hot spots that receiveradio frequency signals from personal devices that are within rangeand/or try to connect to the router and/or hot spot. The PDPS 40 canmeasure radio frequency emissions in an area and detect data, such asfrom a pico-cell, e.g., small cellular device typically covering a smallarea that is used to extend coverage to indoor areas where outdoorsignals do not reach well.

The basic notion is that a combination of these sensors can identifyspecific people to various degrees of accuracy without necessarily usingvideo/audio and facial/voice recognition. A weight sensor in a carpetcould identify someone 200 lbs., for example. Temperature changes in aroom and heat signatures of individual could be used to identify suchpersons as well as well as other objects.

Motion/movement is detected by motion detectors. Typically a motiondetector can be placed in a room, the signals from conventional types ofmotion detectors can be used to detect presence in a room. In addition,several motion detectors whose signals are processed in unison can beused to detect motion towards a room, into a room, and from a room.These motion detectors can detect motion with a purpose, e.g., the rateand speed at which an object is moving, as well as direction. The PDPS40 can study a person's motions in detail, and develop a signature ofthat person that can be used to recognize the person.

By combining some of these technologies, the PDPS 40 can detect presenceand yet maintain a notion of privacy. In some embodiments, discussedbelow, data leaves the person's premises, however, if there is a fullstreaming audio of conversations (or video of movements) the personmight not be so keen about sharing such data. The PDPS 40 uses acombination of lower-cost technologies (or in some instances higher costbut less intrusive technologies) and signals from these types of sensorson their own may not raise significant privacy concerns.

Another way to detect movement in a room is by sensing/measuringvariations in r.f. signal strength in a room from signals captured by ar.f. receiver in the room. Signals emitted by r.f. devices, such asphones, computing devices, r.f. tags, etc. carried by persons may varyas persons enter/exit the room. Such variations of signals would bedetected and processed to determine location. The extent to which r.f.would be used is primarily based on the extent to which a person desiredto be intentionally tracked. A person bringing an r.f. device(s) into aroom will cause fluctuations in r.f. signal strength.

The PDPS 40 takes the received sensor data and processes the sensor datathrough processing algorithms to detect specific presence. Specificalgorithms that accomplish detection of specific presence are discussedbelow. Based on whether there was a detected specific presence, the PDPS40 may generate data that is added to one or more existing profiles orto a new profile(s).

The PDPS 40 can be configured to determine when a certaingroup/set/collection of people are together. The PDPS 40 tracks (andrecords) how much time an individual uses to perform a task, such as howmuch time a babysitter actually spends with the baby or children. ThePDPS 40 identifies presence (or specific presence) of multiple people toprovide records as evidence of foul play (proving someone else was inthe room when something happened, or actually identifying the personthat perform a specific act.)

Personal Profile Records

Referring now to FIG. 4A, a diagrammatical view of a personal profilerecord 140 with associated movement flow records 160 a-160 n (generally160 discussed in FIG. 4C) is shown. The depiction of a personal profilerecord 140 is exemplary. The personal profile record 140 includes asystem generated profile record ID 141, store for personal identifyinginformation 142 such as a person's name or a person's role in thepremises, e.g., father, mother, etc. The personal profile record 140also includes feature data store 143 that is captured by sensors andused to identify the person to the PDPS 40 (presence detection andprediction system 40), without any identifying action undertaken by theperson. This feature data store 143 can store various information, suchas feature data from facial recognition processing, that could beemployed where cameras and appropriate hardware installed in thepremises or the features can be obtained from other less expensive andless intrusive sensors, appropriately labeled, as discussed above. Forexample, weight detectors can be installed in the premises and othersensors as well as any of the sensors or combinations of sensorsdescribed above can be used. Another technique to detect specificpresence is the presence of an electronic device with a specific IPaddress that is captured by a network hotspot. All of this data can beconsidered feature data that is captured in the store 143 of thepersonal profile record 140.

Each personal profile record 140 can be associated with one or moremovement flow records 160 (FIG. 4C) through storage 144 of movement flowID's. In addition, each personal profile record 140 will have associateddetected historical patterns store 146 (discussed below).

Sensor Records

Referring now to FIG. 4B, a sensor record 150 is also shown. Thedepiction of a sensor record is exemplary. The format of such a record150 will vary according the system and type of sensor. The sensor record150 is a compilation of “raw” and/or processed sensor data from aspecific sensor. Sensor records 150 can be continuously sent to the PDPS40 that filters out duplicate sensor records 150 that contain nosignificant information or sensor records 150 can be sent when aparticular sensor actually detects something of interest, e.g., adetected entity interacting with the particular sensor. Sensor recordsin some instances can be requested by the PDPS 40 sending one or moreparticular sensors messages to download sensor records to the PDPS 40.

As shown in FIG. 4B, a sensor record 150 includes a sensor record ID 151and information such as a sensor ID 154 that identifies the sensor thatcaptured the data, the time 155 and date of the sensor record 156, and asensor message 158, e.g., raw data generated by the sensor and/orprocessed data from the sensor, if the sensor has such processingcapability. Other information may be captured such as IP address of thesensor, etc.

Movement Flow Records

Referring now to FIG. 4C, a movement flow record 160 is shown. Themovement flow record 160 is a structure that records a flow of adetected entity (or in the more complicated cases detected entities)interacting with a series of sensors over a period of time. The PDPS 40produces a movement flow record 160 upon the PDPS 40 receiving a sensormessage(s). The movement flow record 160 can include a movement flowrecord ID store 161, a structure 162 that stores one and likely severalreferences to sensor records by sensor record ID, and in someimplementations stores the references as an ordered list, ordered bytime of when each sensor was encountered (or message received). Themovement flow record 160 includes storage for information such as astore 164 of pointers to one or more sensor record ID's, a store 163 fora system generated, movement flow record ID and a store 166 for time anddate.

For the relatively straightforward example of producing a movement flowrecord 160 for a single entity, the PDPS 40 simply filters as neededsensor records 150 (to eliminate duplicate records) and generates arepresentation (e.g., a directed graph, a graph, a structure, an orderedlist) of sensor records ID's as received by the PDPS 40. If the PDPS 40can determine a specific, identifiable entity likely corresponding tothe received sensor records, the PDPS 40 includes that entity's profileID in the movement flow record 160. Otherwise, the PDPS 40 populates themovement flow record with a system generated ID.

For a more complicated situation where there are multiple entitiespresent in the premises and detected at the same time, the movement flowrecords that are produced may have sensor records that belong to several(i.e., more than one) entities. The PDPS 40 may filter these movementflow records to split these movement flow records into individualmovement flow records for each detected entity. The raw data in therecords can be analyzed to help split the records, as discussed below.

Historical Pattern Records

Referring now to FIG. 4D, from movement flow records 160 and profilerecords 140, historical pattern records 170 are produced. An exemplaryformat for a historical pattern record 170 is shown in FIG. 4D andincludes the Profile ID store 172 and/or System Generated ID 173, eachof which links or associates the historical pattern record 170 to aspecific individual (or in the case of System Generated ID to theprofile that matches the System Generated ID). Alternatively, thehistorical pattern record could include some or all of the data from therespective profile records, as the historical pattern record can takemany forms.

The historical pattern record 170 also includes the detected pattern,which in this case is a Graph ID field 175 that points to a graphstructure, and can include a Data Field that indicates when (e. g.time/days of the week, seasons, etc.) the historical record is relevant.The historical pattern record 170 is produced by the PDPS 40 to providea record of a detected activity pattern for a specific entity (e.g.,person). The historical pattern record 170 is determined by the PDPS 40by analyzing the specific person's interactions with the premises, ascaptured by a series of sensors within the premises. Various techniquescould be used to build the historical pattern record, from resultsobtained from machine learning (pattern recognition) techniquesdiscussed below.

In one technique, the PDPS 40 produces the historical pattern recordfrom movement flow records according to a rule. Thus, the historicalpattern record in addition to the Profile ID and Graph ID will include aRule ID store 174 (the rule from which that historical pattern recordwas processed). Data can also be stored in the historical record patter170 in data field 178. The data can be various such as indicators whenthe pattern is valid, e.g., days, times, etc. with optional orsupplementary data. As the historical pattern record will reference oneand likely several sensor IDs one way to represent the historicalpattern record is a graph structure. The nodes that represent the sensorID's are associated with other information such as time and date of whenthe sensor was encountered and the sensor ID. A graph is a datastructure that includes a set of nodes (vertices) and a set ofconnecting lines (edges) that establish relationships (connections)between the nodes. In the case of the historical pattern record theconnections are generally unidirectional temporal indicators of theorder in which sensors are encountered in the pattern. A graph G may bedefined as follows: G=(V,E), where V is a finite, non-empty set ofvertices and E is a set of edges (links between pairs of vertices). Whenthe edges in a graph have no direction, the graph is called undirected,otherwise it is called directed. In general, information is associatedwith each node and edge of the graph.

Using Information from Sensors to Produce Historical Pattern Records

Referring now to FIG. 5A, the following describes a process 180 forproducing an historical pattern record 170 that is represented by agraph construct. In this example, a pattern is produced as the result ofexecution of a historical pattern rule that forms, e.g., a path, betweena starting point and an ending point of the rule of the objects(sensors) encountered in between the starting and ending points.

A series of movement flow records are received 182 from various sensors.These records optionally can be filtered 184, meaning for a particularrule only certain records received from selected sensors are used forthe particular rule. The process 180 constructs 186 a graph thatrepresents the order and relationship of sensors that were encounteredby the person. In the historical pattern, each sensor encountered isrepresented as a node or vertex of the constructed graph, and eachrelationship between two nodes is represented as an edge or link betweenthe nodes. As such, in this example, the historical pattern isrepresented as a directed graph that is ordered and can be searched. Thehistorical pattern record is populated with data sufficient to associatethe historical pattern record 170 to a profile record 140. Thus, theprocess 180 populates the historical pattern record 170 to include avalue of an identified profile, if determined, in the profile ID 172,populate the Rule ID field 174 and add any optional or supplementarydata to the data field 178. The historical pattern record 170 is alsopopulated with the reference to the Graph ID 175 of the determined graphstructure that stores in this embodiment the determined historicalpattern.

Referring now to FIG. 5B, an exemplary graph structure is shown. Thisgraph structure will be used in the example discussed in FIG. 5C. Thegraph structure represents an historical pattern produced for anindividual. The graph structure in this example is composed of ninenodes that represent sensor ID's A, D, C, F, Z, K, Y, R, and Q. Thesesensor ID's have no special meaning but this unordered listing is usedto illustrate that sensors can be encountered in any order. The nodes ofthe graph represent the sensors and are connected by edges that containinformation regarding the relationship between the connected nodes.

From the movement flow records, the PDPS 40 constructs the graph fromthe sensors that are encountered in the pattern. Each sensor is assignedto a single node on the graph. The nodes of the graph are connected withedges that in this example correspond to relationships between thesensors as encountered by the person. Each edge contains informationabout connectedness, that is, the relationship between connected nodes.This information can take various forms. For example, one suchdescription in one graph could be the time or day and/or season that aparticular node is connected to a different node. The description couldinclude other information, as appropriate between one node and theconnected node. Since each node generally has one or more edges thatconnect to one or more other nodes, a description of the relationshipbetween nodes, can be obtained by accessing the construct represented ina data structure that stores the information defining the relationshipsamong nodes.

Referring now to FIG. 5C, processing 200 of movement flow records 160 isshown. The PDPS 40 (presence detection and prediction system 40)collects movement flow records overtime. Generally, the PDPS 40 can keepa fixed number of days' worth of records, e.g., 7 days, 14 days, a monthetc. or a fixed number of records. In some embodiments, the PDPS 40 cankeep a very large number of movement flow records, but in general, thePDPS 40 continually updates the movement flow records by producing newmovement flow records and associated these new records with acorresponding profile and deleting outdated movement flow records.

In general, the PDPS 40 receives 222 a movement flow record, determines224 whether the movement flow record includes significant informationregarding a significant action. To determine whether a movement flowrecord includes a new significant action, the received movement flowrecord is compared to stored movement flow records from that Profile ID(or system generated ID). The values in the stored record (e.g., atleast the series of sensors encountered) are compared to correspondingvalues in the new record. If the received record and stored recorddiffer by an empirically defined amount 227, the movement flow record iseither used to replace 226 an existing record or is stored as a newmovement flow record 228.

For example, if the new movement flow record for a specific profile fora different day of a week as a stored movement flow record has theentity corresponding to that profile ID interacting with the samesensors in the stored movement flow record, but on different days of theweek, the PDPS 40 can determine that the new record includes a newsignificant action and stores the new movement flow record.

However. if the new movement flow record for the specific profile forthe same day of a week as a stored movement flow record has the entitycorresponding to that profile ID interacting with the same sensors inthe stored movement flow record, the new record can be discarded, but acount can be kept (e.g., in the record) indicating that the samemovement flow was again encountered on that day. Otherwise, the PDPS 40determines if it should replace an existing movement flow record as thecurrent one has updated data but no new actions, and if so, deletes theexisting record, and if not adds the new movement flow record to storageand processes that record with other records to either generate orupdate an historical profile.

Example of Producing Historical Pattern Records

Referring now to FIG. 5D, in this example 230 the PDPS 40 seeks to learna pattern of what persons do when they first come home during thework-week from previously produced and stored movement flow records. ThePDPS 40 process these previously produced movement flow recordsaccording to various pre-defined or user defined rules.

The rules can be organized in a hierarchy that involves an upper levelrule that invokes lower level rules.

An example of an upper level rule is:

-   -   <routine of person <X> coming home in the evening>.

This upper level rule can be predefined on the PDPS 40. Upper levelrules can be rules that are used to establish a specific pattern such asthe rule above. Lower level rules can be associated with various sensors(either individually sensor records or groups of sensors movement flowrecords). Lower level rules are invoked according to the specific upperlevel rule. Thus, for the above upper level rule, the followingexemplary rules could be used to determine this pattern from sensorrecords stored by the system.

Assume that the premises is occupied by several persons A, B and C. Theactivities of each of these persons can be processed by the routine ofperson <X> rule where “X” seeks to learn an activity pattern forentities A, B or C from, e.g., two different sensors, a weight pad ateach door and an r.f. detector at each door. In this example, thepremises has two entry points (doors) from the outside, with weight padsat each entry point. In this example, there are motion detectors inseveral rooms and a near-field r.f. detector at or near each door.

Many sensor records and movement flow records from sensors in thepremises are accumulated over various times. The process receivesmovement flow records, which initially are assumed, as having nospecific identifying data.

To process the upper level rule, the PDPS 40:

-   -   <retrieves movement flow records that include detected cell        phone at the front door>    -   <retrieves movement flow records that include a detected cell        phone at the side door>    -   <retrieves movement flow records that include a weight measure        at a weight pad at front door>    -   <retrieves movement flow records that include a weight measure        at a weight pad at side door>    -   <produces intersections of movement flow records according to        common detected cell phone and common detected weight at common        doors>    -   <groups of intersecting movement flow records may correspond to        A, B, C>    -   <assigns A, B and C to grouped movement flow records>

In a subsequent pass, the process executing on the PDPS 40 learns whateach of A, B, C does after entering the premises. For example, focusingon person “A,” the PDPS 40:

-   -   <filter movement flow records to retain only records that have        A's weight +/_% deviation>    -   <for given day sort movement flow records of A's weight in a        chronological encountered order>    -   <stores sorted movement flow records of A's weight for the given        day as a pattern for the given day>

Thus, initially the PDPS 40 detected a person “A” was approaching thefront door of the house by the person's client device, e.g., a smartphone being detected by a hotspot. The PDPS 40 next detected that theperson entered the front door and receives a weight measure from the padsensor. These two sensors can be used to filter out other movement flowrecords that do not have this weight and cell phone combination andremaining records are sorted to construct a chronology of activity for Aon a given day for a given time period. The chronology is determinedaccording to the day/date/time data in the movement flow records.

At any point, this pattern for A with sufficient identifying informationcan be associated with a profile for A that specifically identifies Afrom the profile database. The PDPS 40 retrieves that person's profile,if any, from storage and retrieves that person's personal data.

Thus, referring now explicitly to FIG. 5C, subsequently the PDPS 40determines from received 232 movement flows records when person “A”(whether associated with a specific profile or a system generatedprofile) enters the house through the front door 234, the processretrieves profiles 236, and when the person steps on a rug under whichis a mat that has a weight sensor. The weight sensor sends a message tothe PDPS 40 to inform the PDPS 40 that an object of x lbs. was detected238.

If the PDPS 40 confirms that the detected weight roughly corresponds tothe weight of person A, the PDPS 40 can assume that the object detectedis person A and processes detected events into an output correspondingto the rule 242. Otherwise it processes another profile record 239.

As a more concrete example, assume that person A's initial routine thefirst time the PDPS 40 processes this rule, detects the followingpattern of behavior.

A's cellphone is detected on Monday, a workday at 6:05 PM.

A person entered via the front door triggering the weight sensor pad.

A person traveled via a hallway to the kitchen a short time after.

In the kitchen, a person turned on the kitchen light a short time after.

A person in the kitchen, 2 minutes later opened the refrigerator.

A person then walked from the kitchen back through the hallway to theliving room.

A person closed the drapes in the living room.

A person turned on a Bluetooth controlled audio device.

In the living room A person remained for an extended period of time.

The PDPS 40 processes 242 these series of events as the <routine ofperson <X> coming home in the evening>, as mentioned above. Thus inoperation, the PDPS 40 constructs 244 a graph that represents thedetected pattern corresponding to person A's historical pattern whenperson A comes home. The PDPS 40 detects 246 this historical pattern asa generally regular occurring pattern that occurs on a regular basiseach workday around that time of day, and thus saves this as a detectedpattern in or associated with person A's personal profile (FIG. 4B)using the cell phone detection and weight as delineating “A” as theperson detected.

The graph constructed is diagrammatically depicted in FIG. 5B and showsa node A representing sensor A (a near field r.f. detector) as thestarting point of the person approaching the house. Node A has an edgeconnected to node D representing the weight sensor at the front door.The graph shows that node D is connected via an edge to a node Crepresenting a motion sensor in the hallway. Node C is connected via anedge to node F representing another sensor F in the kitchen. In thekitchen, the kitchen light node Z (sensor Z) has an edge connecting toNode F and the refrigerator contact, Node K (sensor K) has an edgeconnecting to Node F. This person's travel from the kitchen back to thehallway is captured by the second edge between Node F and Node C. PersonA traveled from the hallway Node C to the living room Node Y (where amotion sensor detected travel). There are also edges connecting Node Yto Node Q and Node R, representing the closing of the drapes Node Q andturning on of a Bluetooth controlled audio device sensor Node R. As theperson remained in the living room for an extended period of time, thePDPS 40 can treat this as an endpoint for this processed rule.

Similar processing occurs for B. However, the processing need not beidentical. For instance in this example, B has a weight that differsfrom the weight associated with A and B does not have a client devicethat can be detected and generally enters through the side door.

An example of processing the above upper level rule <routine of person<X> coming home in the evening> for B is as follows:

-   -   <retrieve movement flow records that include detected cell phone        at the front door>    -   <retrieve movement flow records that include a detected cell        phone at the side door>    -   <retrieve movement flow records that include a weight measure at        a weight pad at front door>    -   <retrieve movement flow records that include a weight measure at        a weight pad at side door>    -   <sort movement flow records according to common detect a cell        phone and common weight at common doors>

The PDPS 40 can vary processing according to how much profileinformation is present for an entity. If for example, the PDPS 40 has aprofile on B and by accessing that profile determines that B does nothave a cell phone, then the PDPS 40 can skip the rules that detect acell phone, and apply the lower level rules that retrieve movement flowrecords with weight measures at the front and side doors and sort therecords looking for B's weight or common weight.

-   -   <sort movement flow records according to weight at front door>    -   <sort movement flow records according to weight at side door>

The PDPS 40 can try to use another sensor signal to identify B. Forexample, any one or more of the sensor types mentioned above, whichprovide unique sensor signals that can be correlated to a specificindividual could be used with the weight sensor. Examples include LIDAR,(of course audio and video), temperature and chemical sensors,heartbeat, retina scans, etc. Similar processing would be performed bythe PDPS 40 using one or more of these other sensors in order to improvedetection of B. Additional sensors of course could also be used toidentify A.

In any event, in subsequent passes, the process executing on the PDPS 40learns what B does after entering the premises, in a similar manner asdone for person “A,” by the PDPS 40, as described above.

This processing is iterative and continually refines characteristics andpatterns for each of the entities A, B and C and constructs refinedversions of the graph to represent detected patterns corresponding toperson B's historical pattern when the person B comes home.

Similar processing occurs for C. Again, the processing need not beidentical to that of A and B.

When A and B or A and C, etc. enter the premises concurrently,individual sensor signals will be produced as a result of each entityinteracting with the sensor. Thus, for A the PDPS 40 will most likely beable to detect unique weight signals (especially if they enter separatedoors) or if they enter from the same door will detect separate signalsunless they step on the pad at the same time. The PDPS 40 can thus trackeach of these individuals based on the processing above.

In some instances, especially as the number of persons in an area growsto a large number the PDPS 40 will likely produce movement flow recordsthat have combined information from several individuals and theserecords may not be useful and ultimately will be cleared from thesystem.

In order to execute graph-based processes in a computer severalrepresentations of a graph may be used. Examples of such representationsinclude Object oriented representations where there is a structure foreach vertex (representing whatever information is to be stored), andanother structure for each edge (with pointers to the two vertices towhich it connects). The object oriented representation may include alist or array of structures for each of these objects. Anotherrepresentation is an adjacency list representation, where each vertexkeeps a linked list of the neighboring vertices. The edges are typicallynot included. This representation makes it easier to find edgesconnected to any particular vertex. Another representation is anincidence list that combines the adjacency list and the object orientedrepresentation. To obtain an incidence list, one adds to the objectoriented representation, a list, for each vertex, of pointers to theedges incident to the vertex. Others such as adjacency matrix andincidence matrix could be used. The type of rules can influence whichtype of representation is best to use.

Generalized Rule Process for Producing Historical Pattern Records

Referring now to FIG. 6, the PDPS 40 uses the personal profiles todetect historical patterns 250. In general, for a personal profile, thePDPS 40 retrieves a personal profile 252, and retrieves associatedmovement flow records for that profile. The PDPS 40 has a set of rulesthat are used to detect historical patterns, for example the rulementioned above. In general, the PDPS 40 retrieves a rule 254 andretrieves relevant movement flow records and processes 256 the relevantretrieved movement flow records against the retrieved rule. Each rulewill by necessity involve certain sensors. For example, the upper levelrule above involves sensors that are encountered when A first comes homefrom work, which may be different from sensors encountered when A awakesup on a weekend in the spring. For each rule evaluated, the process 250determines 258 if a pattern was detected. The PDPS 40 stores 259 thedetected pattern as an historical pattern. If for a retrieved rule thesystem does not detect any pattern, the system retrieves 257 anotherrule and repeats the process until the movement flow records have beenprocessed through all defined rules.

Methodology

When the profiling system 42 receives a profile, the system, e.g., PDPS40 stores the profile in a persistent database, e.g., either 52 or 49(FIG. 1). In the profile are personal characteristics and dailyroutine(s). The profile takes into account sensor data from the sensors.A sensor produces sensor data that indicates an event when the sensordetects a sensor condition. A cache memory can be used in the profilingsystem 42 or if the device cannot handle it, the cache can be in thecloud, is used with data aggregated from multiple sensors, e.g., sensorrecords. A section of the database keeps track of sensor data, e.g.,sensor records, from the sensors and manages that data, e.g., sensorrecords. A higher level process determines from the, e.g., sensorrecords whether the data is associated with a particular profile and isworthy of permanent storage as part of that profile or as part ofdetermining historical information.

Once that determination is made, the profile and/or historical patterninformation is updated, as appropriate, the updated profile and/orhistorical pattern information is committed to permanent storage, e.g.,in the remote and/or local database. The cache is cleaned out of thecollected events. This is an ongoing process, as the system collectsevents and builds up the person's profile and historical patterninformation, e.g., the person's routine. A process examines thehistorical pattern information to determine a correlation with a routineconducted in a prior day or that day in a prior week or a prior month orseason, etc. Some analysis may be done in the sensors and that data fromthe sensors may be passed on to processing that is performed locally orin the cloud for the analytics for tracking events and correlate thoseevents over time to determine specific patterns.

Thus the PDPS develops a baseline of the physical characteristics androutines for a person. Overtime however, that baseline will beconstantly changing. The system determines anomalies, a person coming orgoing and as each is detected two things happen, the system determineswhether that anomaly requires a response, (most of the time, no) anddoes the anomaly require the system to reset the determined baseline forthat person. Thus, historical patterns are determined by application ofspecific historical pattern rules, using supervised pattern recognitiontechniques that operate on the movement flow records and profile recordsor even at a lower level by using sensor records and profile records.

Historical patterns need not be determined by application of specifichistorical pattern rules, as discussed above. Rather, patterns can bediscovered using unsupervised pattern recognition techniques thatoperate on the movement flow records and profile records or even at alower level by using sensor records and profile records. The patternrecognition techniques can be applied to discover predictablereoccurring patterns in sensor records and profile records or movementflow records and profile records. Thus, in the example above, ratherthan a rule being retrieved and actions processed according to the ruleby the PDPS 40 to determine what sensors were encountered by the personto satisfy the rule, the PDPS 40 s executes, e.g., unsupervised patternmatching algorithms to process many instances of that person's actionsin the house (movement flow records) at that time to construct a graphthat represents a pattern of behavior. The graph can be labeled andstored.

Many types of learning algorithms could be used. A clustering analysiscan be applied to the data to reduce the data, e.g., to merge movementflow records and sensor records for instance. Pattern recognitionprocesses that use labeled ‘training” data are generally consideredsupervised learning techniques whereas unsupervised learning techniquesgenerally do not have available labeled training data but are used touncover hidden or unknown patterns.

Described was a simplified example. In an actual system many of thesesteps could be done in a different order or left out or other activitiescould intervene. &&&

A Use-Case Scenario:

The PDPS 40 processes sensor data according to the above rule:

-   -   <routine of person <X> coming home in the evening>.

Referring now to FIG. 7, an example of processing 280 by the PDPS 40using the above rule is shown. The PDPS 40 seeks predict a routine ofthe person based on an historical pattern. In this instance, the PDPS 40determines the time of day and day of the week and determines that it isa work-day at 5:50 in the evening. The system will thus reference theabove rule and search its database to determine whether there arehistorical patterns for one or more entities, which are associated withthe particular rule, of what persons in the premises do when they firstcome home during the work-week.

A user person approaches 282 the house. The PDPS 40 detects 284 that aperson is approaching the front door of the house by the person's clientdevice, e.g., a smartphone being detected by a hotspot near that door.The PDPS 40 retrieves 286 that person's profile, if any, from storageand retrieves that person's personal data. The person enters the houseand steps on a rug under which is a mat that has a weigh sensor. Theweight sensor sends a message that is received 288 by the PDPS 40, whichmessage is that an object of x lbs. was detected. The PDPS 40 confirms290 that the detected weight roughly corresponds to the weight of theperson, (from the profile) and assumes that the object detected is theperson. The PDPS 40 processes 292 that retrieved profile, if any, fromstorage or adds a new profile and retrieves that person's personal data.The PDPS 40 retrieves 294 an appropriate historical pattern based onlocation (front door), time of day and day of week. The PDPS 40 willproduce predictions 296 of the person's behavior according to theretrieved historical pattern.

Previously in this example, the system detected a pattern of behavior inwhich this particular person on, e.g., each workday around 6 PM, entersvia the front door, travels via the hallway to the kitchen. In thekitchen, the person turns on the kitchen light, stays a few minutes andthen opens the refrigerator. The person then walks from the kitchen backthrough the hallway to the living room where the person closes thedrapes and turns on a Bluetooth controlled audio device. In the livingroom person remains for an extended period of time.

The PDPS 40 having detected the presence of the person and based on thepattern retrieved, generates predictions of behavior. These predictionscause the PDPS 40 to generate messages 298 and send 300 these messagesto appropriate systems in the house to control specific systems/devicesin the house, in a manner determined by the historical patterns and theprofile. The PDPS 40 receives additional sensor data 302 and generates304 additional control messages based on the additional data, historicalpatterns and profile, as well as storing and updating as necessarymovement flow records and historical pattern records.

Thus, as an example, the PDPS 40 retrieves the pattern and based on thepattern PDPS 40 predicts that the person will immediately walk intokitchen and turn on the light. The pattern PDPS 40 thus generates asignal that controls the lights to cause the lights to turn on. The PDPS40 also predicts that a short time later that person will close thedrapes in the hallway. The PDPS 40 receives signals from sensors in thekitchen and hallway. The PDPS 40 processes these signals and forms anindication that the person is likely headed towards the living room. ThePDPS 40 generates and sends messages to close the drapes and control theBluetooth controlled audio device.

Thus, in this mode of operation, the PDPS 40 processes the historicalrecord to determine what devices/systems the PDPS 40 will control andgenerates and sends these messages in a temporal order based on thehistorical profile and current signals from sensors that detect theperson is progressing though his routine. Thus, above when the PDPS 40received the signals from the sensors in the kitchen and hall way thePDPS 40 processes these to form a decision that the person has exitedthe kitchen and likely is following the historical pattern, as theperson is in the hallway and thus is headed towards the living room. Apersonal profile can be associated with many historical patterns, eitheraccording to specific rules or which are discovered through patternrecognition. Many settings in a house can be controlled such as inaddition to controlling drapes and electronic devices, others includethermostat settings, appliances, doors, lighting, and so forth.

On the other hand if the there is no historical record in the profilethat indicates what happens when the person leaves the kitchen, varioussensors in the kitchen and living room will send signals to the PDPS 40and the PDPS 40 will generate a new movement flow record that may beadded to the profile.

Another rule could be what happens when the person exits the house. Theperson's presence has been removed, as detected by sensors, so varioussystems in the house can go to a state desired when the person is nolonger present, such as doors lock, blinds close, lights are turn off,thermostat settings regulate to a lower temperature in winter, highertemperature in summer and a house alarm system is ARM'ed.

Detect and Recognize Persons Using Inexpensive Sensor Technologies

Referring now to FIG. 8, in some implementations inexpensive andnon-intrusive, e.g., privacy-aware, technologies can be used to detectand recognize presence of specific individuals. Conventionally, use ofvideo and audio technologies with appropriate recognition software,e.g., facial and/or voice, can be used to recognized the presence ofspecific individuals. However, some persons may find such approachesextremely intrusive, i.e., non-privacy aware. In addition, in thecontext of residential intrusion and alarm systems, oftentimes video andaudio technologies are not part of such systems, but have other sensortypes. It would be nonetheless desirable to detect and recognizepresence of specific individuals without using video and audiotechnologies.

In this context, the PDPS 40 has a presence detection mode 250. In thispresence detection mode 250, signals from plural, different types ofsensors are processed and used to recognize the presence of specificindividual(s). In general, the PDPS is configured to examine signalsfrom specific sensors (specified based on location and the type ofsensor element). The types of sensors used exclude video cameras, andmicrophones with circuitry to detect and process speech. The types ofsensors used include r.f. detectors that can detect r.f. energy emittedby devices during operation and especially those devices wirelesslyconnect to a network. These r.f. detectors can be network hotspots,routers, and the like that can capture an IP address from such r.f.devices attempting to connect to the network, detection of devise comingon line or interrogate a device for a device ID, etc.

Microphones can be used to detect sounds as well as voice signatures(may pose fewer concerns) and sound detection poses even fewer concerns,both of which can be used for presence detection. Temperature sensorscan be used through detection of whether the temperature of room haschanged. Similar changes can be detected by humidity detectors thatmeasure changes in humidity caused by a human entering an area.Vibration detectors within a room can be used to indicate motion in theroom. Chemical/electrochemical detectors can detectchemical/electrochemical emissions/changes. Unique emissions can be usedto generate a signature to identify who the person (or animal, e.g.,pet) is by processing chemical/electrochemical sensor data to determinea biochemical signature that may be relatively unique for a person oranimal.

Contact sensors at entry and exit doors (and other places) can be usedto infer human presence. User devices such as, but not limited to, cellphones, generally have GPS transceivers that send/receive location dataof the device (as well as device identifying information). The GPSmessages can be received by r.f. detectors and used to detect movement,rate of movement and direction, especially outside. Weight Sensors suchas pressure or contact pads, can be installed in and under, furniture,carpeting, and flooring in the premises. Other sensors include opticalor retina scanning detectors, ECG Heartbeat detectors that detectheartbeats in wearable computing devices and by using the Doppler Effectprocessing mentioned above, such detected heartbeat signals can beprocessed to estimate motion towards or away from given location. Othersensors that can be used are those that detect changes in light, e.g.,sensors can detect shadows moving across a room or detect whether alight has been turn-on (or off), determine if lights were left on whilepeople are no longer in the room.

The above physical/chemical/optical characteristics can be capturedusing generally conventional sensors. In many installed intrusiondetection (e.g., integrated security, intrusion, alarm systems) many ofthe above sensors are already deployed.

Still referring to FIG. 8, the presence detection mode 250 receives 252signals from various ones of these sensors (as well as other types) asdescribed above. In the presence detection mode 250, the presencedetection mode 250 uses 254 selected sensor types and sensors data tospecifically identify moving objects to enable prediction of a specificindividual object (typically a person or animal).

The presence detection mode 250 will retrieve existing profiles 256 (asdiscussed above). The presence detection mode 250 extracts 258 from theexisting profiles values for specific parameters that correspond to thetypes and specific sensors that are being examined as provided by theselected sensors. The presence detection mode 250 determines 260 ifthere is a sufficiently close match (within a predefined tolerance thatis based on expected variation in a value of the parameter) for any ofthe parameters in the profile. If there is not a match of receivedsensor data to any stored parameters in the profile, the PDPS 40retrieves 262 the next profile, if any, and repeats the extract 256 andcompare 258 functions. If the presence detection mode 250 finds nomatching parameters in any profile, the presence detection mode 250exits (not shown).

Thus, as used herein “close match” (or the like) corresponds to applyingan empirical tolerance or to set out a range of values for some or allof the characteristics detected or sensed by the various sensors, whichwhen compared to an actual reading would produce an indication of amatch if within the range or tolerance or not match if outside the rangeor tolerance. In addition the term “sufficient number of close matches”(or the like) corresponds to a value of the number of matches ofcharacteristics (within the tolerance or range) detected or sensed bythe various sensors that are empirically determined or considered asadequate to produce a decision, such as to provide an identity for anindividual entity.

For example, to detect a specific presence of an individual it may beconsidered sufficient that the sensors detect a specific set ofcharacteristics from sensors such as weight, r.f. hotspot, and achemical signature. The result from an r.f. hotspot may not have atolerance, meaning that a specific device ID would be required, whereasweight could have a tolerance of, e.g., +/−5%. Chemical signature couldhave a tolerance that may be more complex than the tolerance assigned toweight. If only two of the sensor signals are detected, it may beconsidered an insufficient number of close matches and either othersensors would need to confirm or the system would not indicate aspecific presence.

However, if the presence detection mode 250 determines that there areone or more matches of parameters to sensor data, the presence detectionmode 250 determines 260 the number of parameters in the profile thatmatched received sensor data, and stores 264 this determination on aprofile basis. The PDPS 40 determines 266 whether there are moreprofiles, if yes, the PDPS 40 retrieves 262 the next profile. When allstored profiles have been examined, the PDPS determines which profile(s)are the most likely matches to the user, that is, the PDPS produces 268a prediction of the specific presence of an individual.

In the presence detection mode, existing sensor technologies (existingin the sense that these are currently available and/or existing in thesense that these are commonly employed in existing intrusion detectionsystems, etc.) and their outputs are used with the above algorithm inconjunction with profiles generated by the profile system 42 to form aspecific recognition of an individual. For example, a regular motionsensor can detect that there is motion, an audio analyzer in conjunctionwith a microphone can detect if the motion has an audible signature, acamera can capture what caused the motion, but without detailedrecognition or video data leaving the premises, and an IR (infrared)camera can capture an IR signature of the motion and these signals incombination with learned patterns of a specific individual can be usedto form a specific presence prediction of a specific individual.

Set out below is a table that contains in rows, groupings (i.e.,combinations) of different types of sensors whose signals may beprocessed by the presence detection system in a manner that can uniquelyidentify persons and associate the identified persons to specificpersonal profiles.

Sensor Sensor Sensor Sensor Sensor r.f. detector weight r.f. detectorAudio r.f. detector LIDAR Weight sound Video Weight Motion Audiodetector Chemical Weight Chemical Weight ECG heartbeat Weight Opticalscan Weight Heat Weight Motion Weight Chemical detector Motion Weightr.f. detector detector Motion Weight Audio detector ECG heartbeatChemical retina scan Vibration Chemical Weight Motion r.f. detector

As can be seen above depending on the sensor type and the degree ofaccuracy one (for unique types of sensors such as retina scans) orseveral sensor combination may be used. The above are merely examples ofcombinations and of course other sensor types and combinations could beused. Some of the examples include relatively expensive detectortechnologies, such as cameras and LIDAR whereas other combinationsinclude relatively inexpensive or commonly found technologies such asweight and audio detectors.

Referring now to FIG. 8A, determination 268 of specific presenceincludes retrieving 268 a stored matches for a profile that was found inFIG. 8 to have one or more parameters sufficiently close to receivedsensors values. Of course, some parameters in the stored profile areexact matches, e.g., detecting a specific IP address of a user device,whereas others are bounded by an empirically determined range, e.g., aweight value W. For example, the weight value W could defined by a rangeof +/−1% or +/−2% or +/−5% for instance of a nominal weight W′ orW=W′+/−1%. For the retrieved profile, the determination 268 processdetermines 268 b a weighted value for a prediction for the profile andstores 268 c the value, e.g., in association with a profile ID. Theprocess 268 determines 268 d if there are more profiles to consider, ifyes, retrieves 268 e the next profile, and if not, the process 268determines while profile has the highest weight value, and canoptionally compare that highest weight value 268 d to a threshold value.

The PDPS 40 executing the determination of predicted specific presence268 feature (FIG. 8) of the presence detection mode 250 produce results,which can be as in a Table, (below) that stores results of the compareoperation 260 (FIG. 8) shown on a per profile ID bases. Some sensors canbe a binary value whereas others can be a measured value.

Profile sensor Prediction ID A sensor B sensor C * * * sensor n (P) ID 1185 lbs. IP address X N — P_ID₁ ID 2 147 lbs  IP address Y N — P_ID₂ ID5 200 lbs. N Y — P_ID₃ ID 6 — — — — P_ID₄

In this example sensor A is a weight sensor; sensor B is a hotspot andsensor C is a chemical sensor.

For a weighted value prediction (P_ID_(n)) a predicted value (P_ID₁),can be defined for ID 1 as:

P_ID ₁ =w1*sensor A+w2*sensor B+w3*sensor C+ . . . wn*sensor n

Assuming in an example that P_ID₁ is the highest. P_ID₁ is thus comparedto a threshold and if P_ID₁ satisfies the threshold condition, theentity defined by profile P_ID₁ is predicted as the specific identifiedpresence.

One of the more difficulties with some implementations of intrusiondetection and alarm systems is the occurrence of false alarms. Within apremises, a sensor may fire and that sensor may raise an alarm thatcorresponds to an alarm condition that requires notification of theauthorities to investigate. Oftentimes, video and audio technologies maynot be incorporated or available at the point of the detected intrusionto validate the presences of a true alarm condition in the premises. Thesystem correlates data from multiple sensors to provide a context into adetection to have the system more accurately determine whether an actual‘alarm’ situation exists.

In the context of residential use, a motion sensor is tripped, whichabsent any other information could cause an alarm to be asserted becauseof a detected presence or a temperature detector can detect that thetemperature of room has suddenly changed or a contact switch can asserta signal indicating that a window has been opened. In the securitysystem, the PDPS 40 applies intelligent analysis on whether an alarmshould be asserted based on either the motion sensor being tripped orthe temperature detector detecting a rise in temperature of the room orthe contact switch asserting a signal indicating that a window has beenopened. The PDPS 40 seeks to confirm a potential intrusion byreferencing two or more other, different types of sensor signals forconfirmation and also retrieves data pertaining to the asserted sensors.

For example, one sensor in a room can also be a sensor that is sensitiveto light, and can detect that the sun is shining into the room. The PDPS40 can retrieve data that indicates that the detected temperature changeis the result of normal cycling of climate control systems in the house,or that the PDPS 40 had previously detected via an audio sensor or froma signal from a thermostat and/or furnace that a furnace in the premiseshas turned on. Alternatively, the PDPS 40 can detect that warm air iscoming from a vent. These sensors would tend to mitigate against anincrease in the temperature of the room as being an alarm condition.Other sensors such as weight sensors can be used to determine if a bodyof sufficient weight has been detected in the room. If no weight hasbeen detected in the room or if the weight is that of a pet, the PDPS 40can retire the alarm condition, having determined that the allegedcondition was caused by, e.g., the family pet.

In addition, in the context of residential use, user devices such assmart phones, tablet computers, personal computers and the like, producedifferent r.f. signatures and some user devices have different uniqueidentifiers. The r.f. detectors can be of various types and can bedispersed within a residence or can be a single detector such as awireless router, which detectors detect these different signatures toestablish presence of a person associated with the device. With an r.f.detection of an authorized user device in the premises, and otherconfirmation of the presence of the person, the PDPS 40 can likewiseretire the asserted signal.

Using Presence Information Collected from Sensors for Minimizing FalseAlarms

Referring now to FIG. 9 a unified intrusion detection and PDPS platform300 that collects data from sensors (via sensor interface 352) forproducing alarm events and which uses profile and presence informationto minimize false alarms 320 is shown. Although not explicitly shown inFIG. 9, the PDPS system 40 also receives the sensor signals as discussedin FIG. 2. The unified intrusion detection and PDPS platform 300(unified platform 300) includes in addition to the sensor interface 352a modified intrusion detection panel 16′ that includes a processor board354 that is configurable by a signal from the PDPS system 40 that can beused to validate potential alarm conditions, as discussed below. Theunified platform 300 also includes the profile system 42 and detectionand prediction system 60 portions of the PDPS system 40 as discussed inFIG. 2. As discussed above, one of the difficulties with someimplementations of intrusion and alarm systems is the relatively highoccurrence of false alarms. The unified platform 300 uses signals fromthe inexpensive and non-intrusive, e.g., privacy-aware, technologiesdiscussed above to detect and recognize presence of specific individualsand uses analytics for discerning between occurrences of false alarmconditions and true alarm conditions. For instance, alarm conditions canoccur when a motion sensor is tripped, by a moving object that is notindicative of an intruder, e.g., by a resident or authorized person orby a pet or wind blowing a drape, etc. with the system being armed.Absent other information tripping of the motion sensor could cause analarm to be asserted because the sensor detected a moving object. Theunified intrusion detection panel 16′ via the processor board 354detects a potential alarm event(s) from the firing of various sensors.Upon occurrence of the potential alarm event(s) and prior to issuance ofthe alarm, the panel 16′ signals the PDPS 40 in the unified platform300. The unified platform 300 uses presence information and analysis todetermine whether an alarm should be asserted.

Referring now to FIG. 9A, a process 321 for minimizing the occurrence offalse alarms in intrusion detection systems is shown. In someembodiments, the unified platform 300 receives 322 signals from thevarious sensor types, e.g., the inexpensive and non-intrusive, e.g.,privacy-aware, technologies discussed above to detect and recognizepresence of specific individuals. Conventionally, use of video and audiotechnologies with appropriate recognition software, e.g., facial and/orvoice, can be used to recognized the presence of specific individuals.However, some persons may find such approaches extremely intrusive,i.e., non-privacy aware and these approaches may be more costly.

The unified intrusion detection and PDPS 40 platform, accesses 324profile information regarding occupants of the house, including pets. Asmentioned this information is in profile records and includes some orall personal information such as name, age, weight, gender, height, eyecolor, hair color, etc., as well as type of entity, e.g., resident,frequent visitor, pet, etc. The physical attributes are either inputtedby the user or discerned from sensors. The unified intrusion detectionpanel 16′ via the processor board 354 detects a potential alarm event(s)from the firing of various sensors. Upon occurrence of the potentialalarm event(s) and prior to issuance of the alarm, the panel 16′ signalsthe PDPS 40 in the unified platform 300. The unified platform 300collects 326 presence information on the moving object by processingsignals from sensors, as discussed above in FIGS. 1-8. This presenceinformation is compared to existing profile information 328 and if thereare matches to the existing profile information, the unified intrusiondetection and PDPS 40 platform applies analysis 330 that examines whatprofile was matched, historical patterns, sensors that were tripped,etc. to determine whether to actually send the alarm condition.

For instance, some sensors may send signals indication that a movingobject was detected, however the object detected was the family pet. Ifthe family pet has a profile in the PDPS 40, the unified platform candetermine that this is a false alarm and avoid or delay sendingassertion of an alarm until other confirmation from other sensors isreceived.

Thus, the PDPS 40 applies intelligent analysis 330 on whether an alarmshould be asserted based on the motion sensor being tripped. The PDPS 40rather than seeking confirmation of a potential intrusion by referencingtwo or more other, different types of sensor signals relies on theprocessing of the profile and presence information. The profile andpresence information provides an indication of what triggered the alarm.Of instance, if the profile and presence information as processed by thePDPS 40 determines that the alarm was triggered by movement of a familypet or the presence of authorized persons in the premises the PDPS 40signals the processor to retire the event and an alarm would not beissued as such pet or persons most likely caused assertion of the alarm.

The PDPS 40 would not retire the event and would signal the panel 16′ toassert the alarm when countervailing considerations applied, such as,tripping of other alarms or the presence of unidentified or unauthorizedindividuals or an inability of the PDPS 40 to ascertain that thepresence of authorized entities actually caused the event or that thePDPS 40 cannot otherwise correlate the detected presence to the event.In these instance the PDPS 40 would not seek confirmation of a potentialintrusion after processing of the profile and presence information butrather would signal the intrusion detection system to assert the alarm.

Such intrusion detection system thus would be modified to take intoconsideration in any filter processing of potential alarm conditions therelative importance of presence information as processed by the PDPS 40.This modification can be an override of the alarm event assertion ormerely an input into processing of the event by the intrusion detectionsystem. While the PDPS 40 may retire the event and signal the panel 16′to not assert the alarm, either the PDPS 40 and/or the panel 16′ canstore the event for subsequent assertion based on a confirmation fromother sensors.

For example, one sensor in a room can be a sensor that is sensitive tovibration, and the PDPS 40 can detect the absence of a presence of anyentity in the room. Alternatively, the PDPS 40 can detect that warm airis coming from a vent that moved drapes that caused the motion sensor totrip. The false alarms thus would be confirmed by the absence of apresence of an entity such as through no increases in the temperature ofthe room, no weight sensors detecting a presence in the room, etc.Alternatively, if weight has been detected in the room but is that of apet, as verified by the profile information, the PDPS 40 can retire thealarm condition, having determined that the alleged condition was causedby, e.g., the family pet.

Using Presence Information Collected from Sensors for Simplified UserInteraction

Referring now to FIG. 10, the unified intrusion detection and PDPS 40platform 330 (FIG. 9) also includes a process 350 that simplifies userinteraction with the system based on collected data from sensors andpresence information. The PDPS 40 can be instrumental in simplifyinguser interaction with the intrusion system 12 via the panel 16′. ThePDPS 40 senses user activity and identification of the user to configurethe intrusion system. Specifically for user interaction, the use of aremote sensor can act as a ‘trigger’ event to enable user interactionwith the intrusion detection system 12, such as by voice commands fromspecific persons.

For example, signals from multiple sensors are received 352 by the PDPS40. These sensor signals are processed 354 by the PDPS 40 to detect thepresence of a moving object in a specific location in the premises. Thespecific presence and the location are determined as above, e.g., by theprocessing of FIGS. 8, 8A and by the PDPS 40 determining whatroom/location the user is in by knowing what sensors are in whichlocations in the premises. The PDPS retrieves home “profile” data thatspecifies locations of various sensors, layouts of rooms, equipment inthe home, devices/systems that can be electronically controlled, IPaddresses of such system/devices, etc.

Once the PDPS 40 has determined that an occupant is in a particularroom, the PDPS 40 determines 356 what mechanisms are available in thatroom to communicate with the PDPS 40 and the intrusion detection system.For example, the PDPS 40 determines that only voice input via microphoneis available to interact with the intrusion system 12, so the PDPS 40enables 358 a microphone in that room to listen for key words to beginuser interaction with the intrusion system.

Thus, the PDPS 40 includes the described hardware and a scalablesoftware platform that collects via the sensors environmental data thatare processed by a centralized analytical engine. The analyticalprocessing is based on software algorithms in combinations with thesensor information that generate event/timing messages/signals tocontrol home devices (such as switches, controllers, remotes, smallappliances, entertainment systems, motors, etc.) without the directintervention of the homeowner. An example of this could be: as the homeowner approaches the home, the PDPS 40 detects presence of an entity,determines specific identity of the detected entity as discussed above,and based on the determine specific presence, triggers events thatoperate with various devices/systems within the home, which events arebased on learned habits of that specific entity.

Using Presence Information Collected from Sensors for Replicating ofPattern of Activity, Family Monitoring and Alarm Events.

Referring now to FIG. 11A, a process 370 that uses presence informationcollected from sensors for replicating of patterns of activity is shown.In process 370, the process 370 may receive 372 a request for operatingin an activity replication learning mode over one or more specificperiods of time. The request can be entered via a user interface and caninclude specific times/days of the week and/or seasons or can be aubiquitous request that covers an entire day, week, etc.

The PDPS 40 parses 374 this request, and retrieves 376 historicalpattern records discussed above according to the parsed request.Retrieval of historical pattern records can be on a profile bases orover a period of time basis. The historical pattern records arerepositories of patterns of activity in a house over periods of time.The PDPS 40 can provide overall patterns of activity by linking 378together selected ones of these historical pattern records. These linkedhistorical patterns can be stored and retrieved to replicate activitywhen the PDPS 40 is placed in an activity replication mode (FIG. 11B).

Alternatively, as shown in FIG. 11A, the process 370 can be morecomplex. The process 370 can examine the parsed request to determinewhat actions will be replicated by several individuals over thespecified period(s) of time. Thus the process 370 determines whether therequest requires aggregating 380 of routines of several individuals. Ifso, the process will continue to retrieve profile records 384 andcorresponding historical pattern records of occupants until all profileshave been processed 382. The process 370 stores lists of actions thatcorrespond to a replicated pattern. Replication of action patternsproduces a listing 386 or forms 388 lists of routines for severalindividuals and mergers 390 those lists. The lists include controlactions that are initiated by the PDPS 40 to replicate actions of one ormore of the individuals in the premises. This listing includesinformation such as depicted in the table below:

Device/System ID Action Time Television in child's #1 room Turn on 4:00pm Television in child's #2 room Turn on 4:30 pm Kitchen lights Turn on4:30 pm * * * * * * * * * Television in child's #1 room Turn off 8:00 pmTelevision in child's #2 room Turn off 8:30 pm Kitchen lights Turn off10:30 pm 

The above table is merely a simple listing of possible actions. Forexample, the device system ID can include e.g., a IP address of a devicethat controls the listed device/system, details on formats of messagesand how to interface with the specific device. The PDPS 40 can controlfor instance, lights in various rooms, televisions and other mediadevices, etc. to give an external observer a notion of presence ofhumans in the house. The PDPS 40 thus keeps those historical patternsactive so the house looks lived in even when the occupants are not home.The lists can be labeled according to time/day/season, etc.

Referring now to FIG. 11B, a person configures the PDPS 40 in theintrusion detection system to operate in a replication mode 400. In thereplication mode 400, the PDPS 40 receives a request 402, retrieves 404information regarding the current time and day of the week. The PDPS 40retrieves 406 the lists of actions and executes 408 the actions by thePDPS 40 producing replication of activities according to the currenttime and day of the week (and season). The PDPS 40 applies each or atleast one of the actions from the list to the premises. That is, thePDPS 40 accesses the appropriate stored list (see exemplary table above)retrieves a first control action from the list, initiates the controlaction (by sending the appropriate control message/signal to the device)at the appropriate time and day of the week, etc. to replicate thataction in the premises.

The PDPS 40 also detects patterns to aid in configuring the intrusionsystem. The PDPS 40 retrieves historical pattern information to aid inconfiguration of the intrusion system. For example, the system willperiodically retrieve historical pattern information records, for acurrent time/day and determine whether the retrieved historical patterninformation bears any relevance to configuration of or control of theintrusion system. As an example, the PDPS 40 can execute a rule to ARMthe intrusion system. The rule can be “if occupants turn of < > light(s)in a < > room(s) at or around a < > time cause the PDPS 40 to ARM thesecurity system. The parameters of the specific lights, specific room(s)and time are derived from the retrieved historical pattern informationfor that time. The retrieved historical pattern information can also beused to detect unusual patterns that may provide context for raisingsecurity awareness or an alarm.

By replicating stored patterns of activity, the PDPS 40 can enable ahouse to appear to an outside observer as being occupied by individuals,when in fact the house is unoccupied. As the PDPS 40 continuallycollects information regarding normal behavioral patterns in a premises,the PDPS 40 uses this historical information to replicate one or more ofthese patterns to present an impression of human presence and activityin a home, when it fact there is none.

Referring now to FIG. 12, the processing discussed above can be tailoredfor monitoring 420 specific family members such as children and theelderly, as well as family pets, and so forth. By combining activityrecords and location data the PDPS 40 uses presence information todetermine if there is adequate activity, and uses location data todetermine if individuals are in locations where they should be or are inlocations where they should not be. This process can include processingwith respect to family pets as well. The monitoring process 420 receives422 sensor data from the sensor devices and the PDPS 40 establishesidentity of a specific person in the premises, as discussed above. ThePDPS 40 retrieves 424 information regarding requirements on individualsin the premises and processes sensor data from one or more of the sensordevices against the requirement. The PDPS 40 forms a message regardingthe requirement for the particular user and sends the message to a userdevice. For example, the PDPS 40 can detect through the sensor data thata specific family member such as child is in the basement of a house,which has been defined by a parent as off limits to the child. The PDPS40 can generate a message that is sent to the parent's smartphone forinstance indicating that sensors in the basement detected the presenceof the child in the basement.

Referring now to FIG. 13 techniques 440 for detecting alarm events inthe unified intrusion detection and PDPS platform 300 include collecting442 data from sensors and producing by the PDPS platform 300 from thecollected sensor data, presence information. The above mentionedinexpensive and/or non-intrusive, e.g., privacy-aware, sensortechnologies are used to detect and recognize presence of specificindividuals as discussed above. The profiles of each of theseindividuals are stored in a database and the PDPS platform 300 producesa listing of individuals expected to be in the premises at anyparticular time. While conventionally, use of video and audiotechnologies with appropriate recognition software, e.g., facial and/orvoice, can be used to recognized the presence of specific individuals,some users may find such approaches extremely intrusive, i.e.,non-privacy aware.

The PDPS has a presence detection mode that can detect the presence ofand ‘identification’ of people in the premise. The PDPS, upon detecting444 presence of a moving object, e.g., a person retrieves 446 profilesof ‘expected’ people. The presence detection system also extractsfeature data from various sensors and compares parameters in theretrieved profile records 448 to feature data from various sensors todetermine 450 whether there are any matches of features of that personwith corresponding features of expected people. If there are no matchesto any profiles the PDPS 40 can build a new profile. For a matchingprofile, the process 450 determines 454 whether a person isunauthorized, and identifies 456 when an unidentified person has beendetected and raises an alarm 458.

However, rather than the PDPS performing this processing, the PDPS cansend the profiles of expected people and raw sensor data collected froma detected individual to a third party monitoring service and the thirdparty monitor service will attempt to match features from the raw sensordata with features in the profiles and raise an alarm if there were nomatches. In this presence detection mode, signals from plural, differenttypes of sensors are processed and used to recognize the presence ofspecific individual(s). The PDPS receives signals from various sensors,of various types as described above. In the presence mode, the PDPSsuses selected sensor types and sensors data to specifically identifymoving objects to enable prediction of a specific individual object(typically a person or animal).

The PDPS will retrieve existing profiles (as discussed above). The PDPSwill extract from the existing profiles values for specific parametersthat correspond to the types of data provided by the selected sensors.The PDPS determines if there is a sufficiently close match (within apredefined tolerance that is based on expected variation in a value ofthe parameter). Table 1 above of sensor types are exemplary of sensorsthat can be used in unison to detect specific presence of individualmoving objects.

An example monitoring station can be a single physical monitoringstation or center in FIG. 1. However, it could alternatively be formedof multiple monitoring centers/stations, each at a different physicallocation, and each in communication with the data network. The centralmonitoring station 18 includes one or more monitoring server(s) eachprocessing messages from the panels and/or user devices of subscribersserviced by the monitoring station. Optionally, a monitoring server mayalso take part in two-way audio communications or otherwise communicateover the network, with a suitably equipped interconnected panel and/oruser device.

The monitoring server may include a processor, a network interface and amemory (not shown). The monitoring server may physically take the formof a rack mounted card and may be in communication with one or moreoperator terminals. An example monitoring server is a SURGARD™ SG-SystemIII Virtual, or similar receiver.

The processor of each monitoring server acts as a controller for eachmonitoring server, and is in communication with, and controls overalloperation, of each server. The processor may include, or be incommunication with the memory that stores processor executableinstructions controlling the overall operation of the monitoring server.Suitable software enabling each monitoring server to authenticate usersfor different security systems, determine whether a requested controlaction can be performed at the security system based on the location ofa user device from the request is sent, or to perform other functionsmay be stored within the memory of each monitoring server. Software mayinclude a suitable Internet protocol (IP) stack andapplications/clients.

An example user device includes a display and a keypad and in someimplementations, the user device is a smart phone. The keypad may be aphysical pad, or may be a virtual pad displayed in part of the display.A user may interact with the application(s) run on the user devicethrough the keypad and the display. The user device also includes acamera, a speaker phone, and a microphone.

Structurally, the user device also includes a processor for executingsoftware instructions and perform functions, such as the user device'soriginal intended functions, such as cell phone calls, Internetbrowsing, etc., and additional functions such as user authenticationprocesses for a security system, communications with the security systemand/or the monitoring station of the security system, and/orapplications of the geographical limitations to control actions to beperformed by the security system. A memory of the user device stores thesoftware instructions and/or operational data associated with executingthe software instructions. Optionally, the instructions and the data mayalso be stored in a storage device (not shown) of the user device. Theuser device also includes one or more device interfaces that provideconnections among the different elements, such as the camera, thedisplay, the keypad, the processor, the memory, etc., of the userdevice. The user device further includes one or more network interfacesfor communicating with external network(s), such as the network of FIG.1, and other devices.

Memory stores program instructions and data used by the user devices andservers. The stored program instructions may perform functions on theuser devices. The program instructions stored in the memory furtherstore software components allowing network communications andestablishment of connections to a network. The software components may,for example, include an internet protocol (IP) stack, as well as drivercomponents for the various interfaces, including the interfaces and thekeypad. Other software components such as operating systems suitable foroperation of the user device, establishing a connection andcommunicating across network will be apparent to those of ordinaryskill.

Although certain embodiments of the methods and systems are described,variations can be included into these embodiments, or other embodimentscan also be used. Other embodiments are within the scope of thefollowing claims.

What is claimed is:
 1. A system comprises: an intrusion detection system comprising a computing device that receives data from plural non-video and non-audio sensor devices, the computing device configure to: detect intrusions into a premises being protected by the intrusion detection system; a presence detection system operatively coupled to the intrusion detection system, the presence detection system configured to: identify a current location of a user in the premises; retrieve stored profile data and historical pattern records associated with the profile; retrieve premises profile data that specifies locations of various sensors, layouts of rooms, equipment in the premises, devices/systems that can be electronically controlled, IP addresses of such system/devices; determine actions that the user is authorized to perform to control the intrusion detection system and other equipment, devices, systems in the premises based on the determined location, the retrieved historical records and the retrieved profile records.
 2. The system of claim 1 further configured to: send actions that control the other equipment, devices, systems, to a user device.
 3. The system of claim 1 further configured to: determine mechanisms available in the current location by which the user can communicate with the presence detection system; and enable determined mechanism in the determined location to communicate with the presence detection system.
 4. The system of claim 1 further configured to: determine identification of the user by stored profile records and received sensor data for configuring the intrusion system.
 5. The system of claim 1 further configured to: determine user activity with the determined user activity providing a trigger event to enable user interaction with the intrusion detection system.
 6. The system of claim 5 wherein the enabled user interaction with the intrusion detection system is by voice commands from the user.
 7. The system of claim 1 further comprising: a plurality of sensor devices that send the sensor data to the system. 